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About: The Getting Down to Facts project seeks to create a common evidence base for understanding the 
current state of California school systems and lay the foundation for substantive conversations about what 
education policies should be sustained and what might be improved to ensure increased opportunity and 
success for all students in California in the decades ahead. Getting Down to Facts || follows approximately a 
decade after the first Getting Down to Facts effort in 2007. This research brief is one of 19 that summarize 36 
research studies that cover four main areas related to state education policy: student success, governance, 
personnel, and funding. 
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This brief summarizes two Getting Down to Facts // technical reports on continuous improvement 
for California: 


Toward a Common Vision of Continuous Improvement for California 
Alicia Grunow, Sandra Park, Heather Hough, Jason Willis, and Kelsey Krausen, September 2018. 


Using Data for Improvement: Learning from the CORE Data Collaborative 
Heather J. Hough, Laura Mulfinger, and Erika Byun, September 2018. 


These and all GDTFIl studies can be found at www.gettingdowntofacts.com. 
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Creating continuously improving education systems could be the antidote to one-off education reforms that 
come and go with little to show for the effort. The strategy has been picking up steam in recent years, urged 
on by the federal Every Student Succeeds Act (ESSA); the California Department of Education; and the Bill & 
Melinda Gates Foundation, which recently announced that it is earmarking 60% of its $1.7 billion investment 
in education during the next five years to support school improvement networks. 


Continuous improvement in education evolved from decades of similar efforts in industry and health care 
that resulted in significant improvement in outcomes and efficiency. What distinguishes continuous improve- 
ment from other reform strategies is its focus on the underlying architecture of the system that is producing 
undesirable outcomes (e.g., low graduation rates, high suspension rates, English learner performance gaps). 
Continuous improvement engages multiple stakeholders (e.g., teachers, administrators, operational staff, 
parents) in disciplined problem-solving to discover, implement, and spread evidenced-based changes that 
work locally to improve student success. Education organizations that have invested in continuous improve- 
ment approaches have been able to provide early examples of the advantages of this approach in education. 


The two reports summarized here describe continuous improvement and what it looks like in practice, ex- 
amine the current state of continuous improvement efforts in California, explain the challenges and barriers 
facing districts trying to get it going, and offer some policy ideas to smooth the way. 
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KEY FINDINGS 


e Continuous improvement is not business as usual. 


e When it’s well understood and appropriately applied, a continuous improvement approach can 
improve education quality. 


e Educators, researchers, and education organizations in California see continuous improvement 
as central to enduring education transformation, but it is still in a nascent stage. 


e Continuous improvement requires an initial significant investment in time and money to make 
it a reality. 


° California’s data systems are not adequate for helping districts monitor progress toward specific 
goals. 


e More training and coaching are needed to build expertise so California can achieve continuous 
improvement throughout the state. 


Summary of Key Findings 


Continuous improvement is not business as usual 


For decades, education reform has been dominated by a series of disjointed efforts that don’t necessarily 
connect with or support each other. Over time, when it became clear that one approach wasn’t going to 
work, another easy fix took its place. Each new initiative was introduced with great hype, but often without 
the support to figure out how to get ideas to work in practice for diverse students in diverse contexts. Results, 
if they were evaluated at all, came at the end of the school year when that group of students was gone. 


Unlike most other education reforms, continuous improvement is a complex, systemwide strategy. It requires 
key shifts from traditional reform approaches. The premise is that the individual people in an organization 
aren’t the cause of current problems, rather that the system is designed in a way that produces the problems. 
Continuous improvement creates a culture of collaboration and experimentation. It provides a structure for 
teachers and staff to identify problems, design interventions specific to those problems, learn from trying 
them out in context, and evaluate their effectiveness while there’s still time to make any necessary changes. 
It is distinguished by assumptions, summarized in Figure 1 (following page), that represent a set of beliefs 
and distinct theory of action about how to make progress. 
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Figure 1: Distinguishing Features of a Continuous Improvement Approach 


ASSUMPTION DESCRIPTION 


Continuous improvement assumes that it is the system and not individuals that 
Systems produce outcomes. produces current outcomes and accordingly focuses attention on system design and 
operation. 


Improvement efforts focus on the processes that produce the outcomes as opposed 


Efforts focus on key processes. : : ; 
ve to focusing exclusive attention on the outcomes themselves. 


Improvement efforts are structured to encourage workers throughout the 

Progress requires collective organization to engage in collective learning about their practice. Data and problem- 
learning and discovery. solving methodologies are used to make assumptions about cause and effect explicit, 
and to test ideas in practice. 


Those directly responsible for implementation of a practice (e.g., classroom teachers) 
are actively involved in learning how to get that practice to work in context. Their 
unique knowledge of the day-to-day work is a form of expertise necessary for 
effective improvement. 


Frontline workers are uniquely 
situated to learn how to get 
ideas to work. 


As effective practices are As effective practices are discovered, they are spread and become standard work for 
discovered, they are spread the organization. These practices are continually updated and adapted to context 
throughout the organization. through local experimentation. 


When it’s well understood and appropriately applied, a continuous improvement approach 
can improve education quality 


In recent years, continuous improvement has been gaining widespread attention in education. Entire school 
districts, charter management organizations, and other groups of public schools brought together by educa- 
tion organizations are turning to continuous improvement to address intransigent and complex problems in 
both instruction and management. 


Some of the school districts that have invested in continuous improvement are beginning to see results. 
For example, the School District of Menomonee Falls, Wisconsin, reduced middle school suspensions from 
283 to 60 and increased Advanced Placement participation more than three-fold within a few years after 
beginning its transition to continuous improvement. Between 2009 and 2014, Fresno Unified School District 
increased its graduation rates from 69% to 79%, and its college eligibility rates from 32% to 48%, as mea- 
sured by California’s A-G course completion rates, which are required for admission to the University of Cal- 
ifornia and California State University systems. Maryland’s Montgomery County Public Schools, which began 
its organizational transformation in 1999, closed its achievement gap during the next decade. By 2009, the 
proportion of students successfully completing Algebra 1 or a higher-level math course with a grade of C or 
better increased by about 54% from 43% to 66%. Gains were especially dramatic for Hispanic and African 
American students, jumping by nearly 188% and 124%, respectively. These examples, and others like them, 
demonstrate that a continuous improvement approach can produce meaningful changes. 


4 | Continuous Improvement: Building System Capacity to Learn 


Educators, researchers, and education organizations in California see continuous 
improvement as central to enduring education transformation, but it is still in a nascent stage 


Continuous improvement is increasingly being embraced by state policymakers, educators, and researchers 
in California as an integral method for improving student outcomes. The California Department of Education, 
for example, uses the term continuous improvement 23 times in the January 2018 version of the state plan 
to comply with the federal Every Student Succeeds Act (ESSA). 


The state’s ESSA plan underscores California’s move toward increasing local control. Specifically, several new 
state policy structures—the Local Control Funding Formula (LCFF), the California School Dashboard, and the 
Local Control Accountability Plan (LCAP) process—have given school districts more autonomy in how they 
allocate resources to support underserved students, how they evaluate student and school performance, 
and how they can choose to intervene when groups of students are not achieving. 


Under these new approaches, district leaders create goals in each of the state’s eight priority areas, and then 
allocate resources strategically to ensure equitable outcomes for all students. The guiding principles of this 
new system are that multiple measures guide educators to focus on the “right drivers” and that local leaders 
are given the resources and expertise to act on what they learn from the data. 


However, since continuous improvement is a systems approach, there is a growing understanding that achiev- 
ing this vision requires improving the way that organizations function. For example, nearly three-quarters of 
education leaders surveyed at a California meeting on continuous improvement in October 2017 agreed with 
the statement: “Districts should be working to become improvement organizations.” However, these leaders 
were in much less agreement with the sentiment that: “We will reach the goal of districts as improvement 
organizations with current conditions in the state” (see Figure 2). 


Figure 2: Survey of Education Leaders Attending Continuous Improvement Convening 


Agreement with the statement: Districts should be working to become improvement organizations. 


Strongly Strongly 
Disagree Agree 


Agreement with the statement: We will reach the goal of districts as improvement organizations with current 
conditions in the state. 


Unlikely Likely 


1 California’s eight priority areas are (a) basic services, (b) implementation of state standards, (c) course access, (d) student achievement, (e) 
other student outcomes, (f) student engagement, (g) parent involvement, and (h) school climate. 
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The state leaders who expressed doubt over whether California districts will reach the goal of becoming con- 
tinuous improvement organizations did so, in part, based on what they see as a lack of understanding about 
what it takes to effectively carry out and sustain it. Even though most educators have heard about continuous 
improvement and have a general idea of what it is, few have made it an integral part of the daily life of their 
schools and districts. 


lf half-heartedly or partially implemented, continuous improvement runs the risk of becoming a surface-level 
reform that doesn’t bring about transformative change. To prevent continuous improvement from becoming 
merely a convenient slogan, the field as a whole needs agreement on what continuous improvement looks 
like when it becomes the expected way of daily life in a district and how to support it at scale. 


Continuous improvement requires an initial significant investment in time and money to 
make it a reality 


Organizations and networks that get results allocate significant time and resources to build improvement ca- 
pacity. These upfront investments have long-term payoffs by enabling educators and administrators to more 
efficiently target their resources toward improvement efforts that work and that are connected to their orga- 
nizational goals. In particular, continuous improvement organizations invest in ongoing training and coaching 
to build improvement expertise. 


In the early stages, organizations often rely on outside experts to help build their improvement expertise. 
Over time, the training gets built into the fabric of the organization and improvement coaching becomes 
embedded into regular role responsibilities. Some of this can be done with district employees after they’re 
fully trained; but at the start, outside expertise is required. This can be a significant barrier for some districts. 
For example, a district leader estimated that a three-year contract with a private consulting and training 
group would cost up to $200,000, which not all districts can afford. And it’s not just money—shifting prior- 
ities to free-up resources takes time. Indeed, the transformation to a continuous improvement district will 
take several years, which runs counter to the traditional accountability model that expects districts to show 
improvements within a single academic year. 


California’s data systems are not adequate for helping districts monitor progress toward a 
specific goal 


In a continuous improvement approach, people at all levels of the system need to know how to decide what 
to measure and how to analyze data to both understand variation in performance among students and to 
evaluate whether a new program or new instructional method is actually improving outcomes. There are 
different phases of continuous improvement, and the data needed at each of these phases are quite dif- 
ferent (see Figure 3 on the following page). In the project-definition phase, data needs are geared toward 
understanding the problem (e.g., which schools or students are underperforming and on which measures). 
During the diagnostic phase, managers evaluate existing processes within that area of interest to diagnose 
potential quality problems and opportunities for improvement. During the intervention phase, individuals 
use data within iterative cycles for implementation and progress monitoring. In the impact phase, leaders 
use data to evaluate the effect of interventions on predetermined performance measures. And finally, in the 
sustainability phase, data are used for monitoring and refining interventions as well as providing feedback in 
order to sustain the improvements. 


6 | Continuous Improvement: Building System Capacity to Learn 


2. Diagnostic 
4. Impact Phase Phase 


3. Intervention Phase 


Note: Adapted from The Victorian Quality Council (2008). P = Plan, D = Do, S = Study, and A = Act. 


Continuous improvement needs both more frequent and deeper data in order to catch problems while 
there’s still time to turn them around. However, California has a very limited data infrastructure. As a result, 
the research conducted for these reports found wide variation in district access to data and capacity to use 
data for improvement. 


The California Longitudinal Pupil Achievement Data System (CALPADS) contains individual student data in- 
cluding demographics, courses taken, discipline, assessments, and graduation. However, it doesn’t provide 
enough useful granular information to support improvement. The data that schools and districts currently 
get from the state are in the words of one district leader “really an autopsy.” They tell what happened the 
previous year, which doesn’t allow the teacher or system leader to use the data to help that group of stu- 
dents because they have moved on. Furthermore, these data aren’t presented back to educators in ways that 
are easy to analyze or understand. 
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As a result, some districts and networks are beginning to build this data infrastructure for themselves. For 
example, a network of districts known as the CORE districts has created a data collaborative to provide faster 
and more detailed information about student and school performance. Nearly 60 school districts that to- 
gether cover almost a third of all the state’s public school students are now part of the collaborative. Districts 
pay a small fee to join the data collaborative, and in return receive access to much more information than is 
available through CALPADS, and with stronger analytic tools. 


Sometimes the CORE data system simply makes better use of data from CALPADS. For example, CORE has de- 
veloped an academic growth metric that provides more information about school performance than what’s 
currently reported by the state. The CORE measure of student growth takes into account an individual stu- 
dent’s prior test history, socioeconomic disadvantage, disability status, English learner status, homelessness, 
and foster care status. With this information, educators can see the impact a school has on student achieve- 
ment compared to other schools with similar characteristics. CORE also reports on-track indicators, including 
high school readiness scores at the middle school level, which are based on predictive analytics that combine 
a student’s GPA, attendance, grades, and suspensions. These indicators measure a student’s risk level at the 
start of high school. 


The data collaborative also adds new measures to state data, including surveys of school climate completed 
by students, parents, and staff; social-emotional learning measures; college-going rates; and college out- 
comes, with additional metrics being developed. 


Member districts also have access to an interactive feature that is available in some states, but not in Califor- 
nia. It allows districts to make comparisons between and within schools, and to analyze how achievement is 
changing over time on the multiple measures contained in the data system. While the data available through 
the CORE data collaborative have filled a need for many districts, they are insufficient to support continuous 
improvement. More frequent data to help actors at all levels of the system are needed to support all the cy- 
cles of improvement, and many districts don’t have access to these kinds of data, nor do they have the time 
and expertise to build the data themselves. 


More training and coaching are needed to build expertise so California can achieve 
continuous improvement throughout the state 


California education leaders interviewed by the researchers reported that one of the barriers to applying 
continuous improvement is a lack of enough training and coaching to build expertise in schools and districts. 
Everyone needs to learn different strategies, tools, and processes that are part of continuous improvement. 
But there are a limited number of districts that can serve as models and illustrate best practices, and a major- 
ity of district leaders surveyed said no outside group is helping them to implement continuous improvement. 


However, California can invest in this training and coaching infrastructure through the state’s System of Sup- 
port, which is designed to help the 200+ school districts that have been identified as requiring differentiated 
assistance. As part of the state’s philosophy of giving local leaders greater authority to determine what’s best 
for local districts, state education officials have tapped county offices of education as lead agencies in provid- 
ing assistance through the System of Support. Although all 58 county superintendents and other county and 
school district staff have received training in continuous improvement, more investment needs to be made. 
The majority of district superintendents interviewed reported that they’re either receiving no support for 
continuous improvement or support that’s more targeted to specific initiatives rather than the systemwide 
focus of continuous improvement. 
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Continuous improvement is rapidly becoming the state’s favored strategy for increasing student achieve- 
ment. In places where it has been successful outside California, districts have been able to invest heavily in 
training and coaching. Cultivating an environment across the state that supports districts’ use of continuous 
improvement methods will require collaboration from state agencies, advocacy groups, technical assistance 
providers, districts, and schools. Educators at all levels, and those who support them, need to develop a 
shared understanding and vision of what a quality education system looks like and what their role is in con- 
tributing to such a system. 


A continuous improvement approach statewide requires a fundamental shift in the mindsets, roles, and re- 
sponsibilities of everyone in the system and how they relate to and interact with each other. In particular, it 
pushes more of the problem-solving and decision-making to schools and districts. In this shifting geography, 
county offices of education and the state must provide districts with the necessary supports. However, county 
offices don’t necessarily have the expertise to provide the support that districts need. In this way, how the 
System of Support is built and executed is critical to ensuring that every school and district across the state 
has the support that it needs. 


Interviews with district leaders show that they support the vision of continuous improvement as outlined by 
California state education officials. However, many are challenged by the lack of a clear definition of continuous 
improvement. Even state education leaders are still working to understand the principles, tools, and processes 
at the core of continuous improvement. Impediments also exist to creating a better data system and building 
the capacity to make use of better data. Finally, continuous improvement requires sustained focus and invest- 
ment over time, which often runs counter to traditional policy pressure to improve virtually overnight. 


California has started to address some of the challenges, but there is more to do to ensure that all systems 
are aligned and that every district and school has access to the tools, resources, and coaching that they need 
to succeed. In this way, state education agencies can play a critical role in setting the vision, developing the 
capacity to achieve it, and building a consensus for the path forward. 


Lead Author Biographies 


Alicia Grunow is coauthor of Learning to Improve: How America’s Schools Can Get Better at Getting Better 
(Harvard Education Publishing, February 2015). Grunow and Sandra Park are cofounders of the Improvement 
Collective, which provides training, consulting, and coaching to help schools and organizations serving children 
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